Bulletin of Electrical Engineering and Informatics 
Vol. 9, No. 2, April 2020, pp. 785~791 
ISSN: 2302-9285, DOI: 10.11591/ee1.v912.1703 O 785 


Implementation of face and eye detection on DM6437 board 
using simulink model 


B. Vijayalaxmi, Kaushik Sekaran’, N. Neelima’, P. Chandana’, Maytham N. Meqdad’, 
Seifedine Kadry° 
'“Wienan Institute of Technology & Science, Hyderabad, India 
°CMR Institute of Technology, Hyderabad, India 
°Al-Mustaqbal University College, Hillah, Babil, Iraq 
‘Department of Mathematics and Computer Science, Faculty of Science, Beirut Arab University, Lebanon 


Article Info ABSTRACT 

Article history: Driver Assistance system is significant in drriver drowsiness to avoid on road 
accidents. The aim of this research work is to detect the position of driver’s 

Received Aug 14, 2019 eye for fatigue estimation. It is not unusual to see vehicles moving around 

Revised 30 Dec, 2019 even during the nights. In such circumstances there will be very high 

Accepted Feb 15, 2020 probability that a driver gets drowsy which may lead to fatal accidents. 


Providing a solution to this problem has become a motivating factor for this 

research, which aims at detecting driver fatigue. This research concentrates 
Keywords: on locatingthe eye region failing which a warning signal is generated so as 
to alert the driver. In this paper, an efficient algorithm is proposed 
for detecting the location of an eye, which forms an invaluable insight 
for driver fatigue detection after the face detection stage. After detecting 
the eyes, eye tracking for input videos has to be achieved so that the blink 


Driver assistance system 
Eye region 
Fatigue detection 


Image rate of eyes can be determined. 
Processing _ 
This is an open access article under the CC BY-SA license. 
Tracking 
CEO, 
Corresponding Author: 
Seifedine Kadry, 


Department of Mathematics and Computer Science, 
Faculty of Science, Beirut Arab University, Lebanon. 
Email: s.kadry @bau.edu.lb 


1, INTRODUCTION 

Driver drowsiness is a major factor in more number of road accidents. According to recent statistics 
annually nearly 1,200 deaths and 76,000 injuries are related to driver drowsiness. Driver assistant system 
is one solution to prevent these on road accidents, which is a major challenge. New technologies need to be 
developed to avoid these hazards on road [1]. The aim of this research work is to develop a prototype 
of driver assistance [2] system. The proposed algorithm focuses on designing a system, which accurately 
monitors the status of driver’s eyes. Drowsiness of driver is first observed in eyes; hence the algorithm 
is designed to monitor the driver’s eyes to avoid on road accidents. Drowsiness detection involves sequence 
of steps such that detection of face, observation of eye movements and eye blink rate. This research work 
focuses on tracking of eyes looking at the face of the image captured using an efficient image-processing 
algorithm. After successful detection of eyes form the capture image, the system is designed to know 
the status of the eyes to detect driver drowsiness. Most of the applications such as facial recognition, accident 
avoidance systems, virtual tools and human identification systems involve the analysis of face images. 
Some specific applications may be listed as: 
a. Driver monitoring systems: Analysis of visual attention is an important parameter to avoid road 

accidents. Hence, monitoring status of eyes and gaze detection can be used to avoid on road accidents. 
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b. Marketing and Advertising: In print and advertising media eye tracking can provide insight into how 
much visual attention the consumer pays in different forms. 

c. Interactive information systems: Eye movement based informational display systems in which the user 
uses his/her eyes as a pointing device such as a mouse pointer to navigate visually selectable items. 

d. Security systems based on iris recognition. 

Clinical diagnostics. 

f. Computer gaming. 

The aim of research work is to detect driver’s drowsiness level. The focus will be placed 
on designing a system that will accurately monitor the status of driver’s eyes. Many existing research work 
is referred while developing this proposed algorithm. Drowsiness Detection involves a sequence of steps 
such as detection of face from the captured images, identification of eyes and eye location and blink patterns. 
This research work focuses on the localization of the eyes, which involves looking at the entire image 
of the face, and determining the position of the eyes, by a self developed image-processing algorithm. 
The system is carried out in following steps. 

a. Face detection 
b. Eye detection 

c. Eye tracking 

d. Fatigue detection 


” 


2. RESEARCH METHOD 

The implementation of status of eyes using MATLAB [3, 4] is the major part of this research work. 
This was done to minimize size and improve efficiency of the code. The entire algorithm is focused 
on locating eyes, which involves looking at the entire image of the face and determining the status 
of the eyes. The eye blink rate-monitoring program can be divided up into a number of sections like face 
extraction, face detection, eye detection, and eye tracking. The performance of each step is highly dependent 
on the steps before itself because the results obtained in each component is passed as an input to the next 
component Figure 1. 


Captured Image 


Skin Color Morphological Connected 
sepmentation -F—- processing —s Component 


analysis 


Output image 





Figure 1. Architecture block diagram 


2.1. Face extraction 

Face Detection: For the captured image, the aim of face detection [5] is to detect the existence 
of face in the image. If the face is present in the image, return the image location with face detection. 
Detection of eye: For the captured image, the aim of eye detection is to detect the eyes. Eye detection 
algorithm is designed to locate the eyes and mark them with a square or rectangle mark. 


2.2. Face extraction mechanism 

Face extraction mechanism involves, identification and detection of face from the captured sequence 
of images. The common approach of face detection is to detect the face region by using the characteristic 
of the skin color. 


2.3. Algorithm for face extraction 
a. Convert the captured RGB image into YCbCr image. 
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b. For each pixel, get the corresponding Y,Cb and Cr values; in order to use them with skin thresholding, 
depending on the Y value for skin color(S50<Y<142; 107<Cb<124; 135<Cr<152) as our 
proposed solution. 

c. Find the different regions in the image, in this step two functions are used the open and fill functions; 
to ensure that only skin areas were selected, then apply connected component analysis, which will 
separate connected heads by using open and close operations 

d. Set some threshold value to separate face and non-face object based on skin region. 

For each region, if height and width are within the range then is a face else it is not a face. 

f. It means depending on the dimension of the object not too much or very small (S00<Area<2500), as in 
our proposed solution. 

Code for face extraction is written in the Mat lab software using the above algorithm. The Simulink 

model is designed using the code written in matlab. Example of face extraction is given in Figure 2. 


o 





Figure 2. Extracted face 


2.4. Eye detection mechanism: 
a. Apply lab transform to the extracted face region. 
b. Apply morphological operations on the output of lab transformed image. 
c. Identify Region of interest (ROT) 
d. Apply region properties on the eye pair 

After successful extraction of face and eye regions from the captured image, eye tracking is 
performed. Therefore, eye location is an indispensable step for eye state detection in the driver fatigue [6, 7] 
monitoring system. At the first frame, eyes must be located before tracking, and to make sure that the correct 
feature is being tracked, and eyes should be relocated periodically during the tracking. Since we are taking 
only the face area, we know that the region of interest is the region containing the eyes. Divide the face 
into 6 quadrants; the region of eyes will be in uppermost 2 quadrants. After the ROI is extracted, lab 
transformation is applied on the eye pair and region properties like Bounding Box analysis are applied 
to detect the eyes through digital signal [8-10]. In the Bounding Box analysis the range of area of the eye 
component to be detected is (100<Ar<910) as in our proposed solution. Example of ROI and Eye detection is 
given in Figures 3 and 4 repectively. Code for ROI and eye detection is written in the MATLAB software 
using the above algorithm. 





Figure 3. Result of ROI Figure 4. Result of eye detection 


2.5. Fatigue detection 
The level of drowsiness is determined using the concept of fuzzy logic. The concept of fuzzy set is a 
class with un-sharp boundaries. It provides a basis for a qualitative approach to the analysis of complex 
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systems in which linguistic rather than numerical variables are employed to describe system behavior and 
performance. It cannot be quantified objectively. Thus, we use computers to apply the fuzzy logic and 
determine the level of drowsiness.The mean and standard deviation are the two input variables used to 
calculate the status of the eyes. The physical state of the eye is the output variable.Results of state of the eye 
is given in Figures 5 and 6 respectively. Code to detect the state of the eye is written in the MATLAB 
software using the above algorithm. The standard deviation for the open eye is (sd<Q.02) as in our 
proposed solution. 





Figure 5. Closed eye Figure 6. Open eye 


2.6. Code generation procedure for DM6437 EVM 

Code Generation procedure consists of MATLAB Simulink Software, CCSv4Software, and 
DM6437 EVM Library tool for the generation of code for the target. Code generation procedure steps for 
DM6437 EVM are shown in below flow chart figure 7. 
Following are the steps to generate code for DM6437: 
a. Open the Matlab/Simulink model, create the Simulink model for the required application 
b. Run the model, create the make file and after creating the make file build the model and set the PSP 
driver path 
If no errors, an .out file is generated. 
Open the CCSv4 software, create the new configuration file and launch configuration file 


Connect the target then load program 1.e., .out file generated from MATLAB/SIMULINK model. 
Run the code and check the output on the screen. 


Open MATLAB Software 


Open SIMULINK MODEL and Create Simulink model for the 
application 


Generate the Out File by Simulating the Simulink model 


Open CCSV4 Software and create the new Configuration 


Oo 20 















model and Build the model 


Connect to the target and load program into the target 


Application is running on DM 6437 EVM 


Figure 7. Code generation Procedure flow chart for DM 6437 EVM [11, 12] 
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3. RESULTS AND DISCUSSION 
This section discusses the results obtained from the SIMULINK model. This section also focuses on 
warning and testing procedures. 


3.1. Results analysis 

In this proposed algorithm, the continuous images are captured and relevant required information is 
extracted. The algorithm is designed to locate the eyes from the captured image and identify the status 
of the eyes. The entire algorithm is developed in MATLAB environment. SIMULINK model is developed 
with the required code generation to gain maximum benefit of code reused. The results of the software 
development are a method of tracking eyes when a video [13, 14] is given as input to the Simulink model. 
The analysis if the algorithm results are given in Table 1. 


Table 1. Result analysis 


S. MEAN STANDERD OUTPUT STATUS OF 
NO. DEVIATION RESULT EYE 
Ls 0.179 0.01742 0 CLOSE 
2. 0.3366 0.2243 ] OPEN 
3. 0.2931 0.0166 0 CLOSE 
4. 0.2747 0.01384 0 CLOSE 
5. 0.4611 0.05534 ] OPEN 
6. 0.3534 0.01691 0 CLOSE 
7. 0.3189 0.03361 I OPEN 
8. 0.1999 0.04755 I OPEN 
9. 0.3624 0.01552 0 CLOSE 
10. 0.3281 0.03449 1 OPEN 


3.1.1. The cases where our algorithm has failed 

In thid section, we will present three different cases where our algorithm failed casel 
shown in Figure 8. Reason behind the failure: The left eye part is not covered in the ROI as shown 
in the figure. The eyebrow that is covered is out of range of the area given. So only one eye is detected 
and the response time [15] could be calculated for the enhancement on the case of failure, case 2 Figure 9. 
Reason behind the failure: The right eye is covered in the ROI but its area is too small to be in the range. 
Hence, it is not detected, Case 3 Figure 10, Reason behind this failure: The right eyebrow that is covered in 
the ROI is in the range of the area given so the eyebrow also detected. 


a. 





Figure 8. Failure casel 





Figure 9. Failure case2 
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Figure 10. Failure case3 


3.2. Alarm 

Alerting the drowsy driver: The main objective of the project is to alert the driver if the system 
decides that he has felt drowsy. To alert the drowsy driver [16, 17] we need an alarm. For this we’ve used 
an audio block at the output. It will give a beep sound if the state of the eye is closed. And if the state 
of the eye is open it will not give any sound. In this way the drowsiness level of the driver can be detected, 
and he can be alerted if he felt asleep [18-20]. 


3.3. Testing 

Testing of each module is performed individually as the system is developed. The testing is done as 
a continuous process as the system is developed. 
a. Matlab 

Matlab code is tested after completion of each module. Each module is written as a part of function 
implementation. The errors can be easily identified through single stepping. The MATLAB code is tested 
for a continuous video, counting the number of correct detections versus the number of incorrect detections. 
This is done twice to check its effectiveness when implemented. 
b. Overall system 

The major difficulty faced is while testing the overall system, which resulted in delay in the system 
from different sections of code. Running the code using live video causes the processes in the region 
of 8 seconds to process a frame from capture to processed image output. This makes the system currently 
unworkable in real-time adding reaction delay to the scenario and the driver would have driven 
for approximately 10 seconds and could have easily gone off the road by this stage. The future work of this 
model is to store the eye extraction values in a public cloud [21-25] to fetch the data anywhere 
from the current place. 


4. CONCLUSION 

A new algorithm for Face and Eye state detection is implemented on DM6437 board using Simulink 
model. The proposed algorithm works on different images under different illumination conditions. Nearly 36 
images from GTAV database 25 and 30 images from local database have been tested in 
the laboratory. The success rate of the proposed algorithm is 89.5%. The purpose of this research work is to 
develop a driver fatigue detection system using vision-based approach based on eye blink rate. The algorithm 
is also tested on video directly connected to the board and results are quite promising, but certain cases 
mentioned in the analysis need to be improved. This paper presents a unique approach of designing Face 
and Eye detection using MATLAB/SIMULINK model which gives an opportunity to experience real-time 
hardware implementation. The analysis report on Video processing application shows that the total time 
required to process the Video is 0.33s, number of block methods for video are 9, number of internal methods 
video are 5, number of model methods are for video are 10, the clock precision for video is 3ns and clock 
speed is 3001MHz. This paper presents an effective implementation of Face and Eye detection applications 
on DM6437 evaluation board for real time applications. 
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